โ NomadNet 1.4.2 released
๐ฌค rns.recipes
top
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ RNS 1.5.0 Testing - Traffic Prioritization & Stability Improvements โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Started by Mark ยท 29d ago ยท 8dd57a7382268096
Page 2 of 6
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-11
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ weird_bleks #11 โ
โ b92ceaa713697cf7 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 28d ago
I've had the same experience as @K8. I've updated my public node to test it and about 12h
ago(after running for close to 10h) i've had connectivity issues. Not sure if it's
related but i suspect it is.
Most nomad pages failed to establish link. Some worked suprisingly, but most didn't.
One particularity i observed was when i tried to connect to rns.recipes via Columba, that
the request failed again and again but i didn't receive a new announce/path, it was still
showing '4h ago'.
This makes me think the path request was blocked somehow, and it never got back to me.
This aligns with the latest changes, and i think somewhere a queue was full so my PRs
never got accepted.
Maybe this happened when the network was being spammed like Mark said, maybe not, who
knows. It did happen with the default settings after Mark decreased the sizes, so there's
that, maybe i should have used higher values for queues.
Unfortunately i didn't actually debug, just observed passively. After restarting the node
it worked better.
โ 1 โ 0 โค 0
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-12
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ Mark #12 โ
โ 8dd57a7382268096 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Thanks for the testing and feedback everyone!
I just pushed another round of fixes and improvements to Aleph, latest on
master now. There was a few mistakes relating to PR processing, and a range of other
bugs fixed as well. I think some of the core pegging might have stemmed from a stupid bug
in the way pending PRs were processed, and that's fixed now.
@K8 (and others), can you try the latest version (and also try dropping loglevel below
LOG_PATHING )? Under insane announce/PR loads, logging all of that is
going to have a significant performance hit, especially on IOPS constrained servers. I
recently refactored all the logging so LOG_DEBUG should now be sufficient
for mostly everything, unless you actually want to see all that path/announce log spam.
โ 3 โ 0 โค 2
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-13
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ KenAKAFrosty #13 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Dug into it some more, findings might be helpful.
Saw a notable, short-term burst after all. It was evenly spread across most of these
public nodes, but struck me as odd that there'd be a unified burst like that.
Kept looking and saw that right around that same time, one of the public nodes flapped.
Also noted about 70% of the destinations are for "lxmf.delivery" and "lxst.telephony",
(~50% the former, ~20% the latter).
It starts to make a bit more sense looking at Sideband on android. The default-on
start_announce config setting means it will re-announce on all interfaces after one
drops. Also, when an interface comes back up, it re-announces again. The rate limiting
there looks like just a single 3.5s cooldown kind of thing.
So it looks highly plausible that what's happening is as follows:
Android Sideband apps are connected to a public tcp/backbone server. Often a few of them.
Then, one of the public nodes flaps, for whatever reason.
When it goes down, all those android Sideband instances re-announce on all interfaces,
including those other public nodes. Genuine, fresh announces which is why they're also
forwarded.
After that, ~5s later, when their interfacce to the flapped node reconnects, it fires an
announce on all interfaces again. The reconnect period is longer than those reactive
announces' 3.5s cooldown, so this would happen pretty reliably.
In fact, if that same flapping public node was itself a client to one of your nodes, then
the behavior makes a lot of sense with what you experienced: If that other public node
flapped for reasons that take that client interface offline for a blip as well (seems
likely), then it would be reconnecting to your node around the same time as all the
android Sideband users reconnect to it, and it forwards those
once-again-fresh-legitimate-announces.
So for each user, a flap can trigger up to 4 announces per user it happens to. There are
the 2 destinations of lmxf.delivery and sometimes lxst.telephony, with the 2nd wave
occasionally getting eaten by the 3.5s cooldown, just depending on where exactly the
timing windows land.
Since these are categorically already known destinations by the time all this happens,
the rush doesn't enter the burst ingress jail. It's only subject to the 2% limit.
Considering the flaps don't happen super often, the recently-added defaults wouldn't
likely stop it either, since it's a 5 announce grace on the per-hour limit of announces
per destination. Even the TCP default bitrate guess of 10Mbps would allow for ~100
transportd announces/second. It would only take like 40-70 android SB users on the other
side of the public node to feel this. Very reasonable that's already happening
Also, a lot of the steady baseline announce flow was app data like this
{"v":1,"n":"Maps","c":0} , meshchatx's map pack beacon thing. It's on by default for
all users and fires one every 15 minutes, even for users with no map packs published. But
a recent commit is changing that setting to be off by default and to not do the announce
unless at least 1 map is published, so that's apparently coming down the pipeline and
should help too.
Anyway, haven't narrowed down the path requests as much, but considering a lot of active
Links could be open on those same interfaces when the flap happens, there's a good chance
that's related, too. The burst I saw also showed a burst of PRs shortly before the
announces, which lines up.
โ 1 โ 0 โค 0
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-14
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ Mark #14 โ
โ 8dd57a7382268096 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Thanks for looking into it Ken, but it's not so much the announce waves happenning as
clients reconnect that I find weird. Those make pretty decent sense, and it's usually
just one or two announces per sub-interface. They do tend to come in a wave, but usually
only around a couple of hundred at a time, not really a problem.
And active links that go down wouldn't cause new path requests when they drop, not in
RNS at least. It's only attempted links that were pending but never established (and only
at the initiator and penultimate hop that would occur, not in any way over the entire
path) that trigger a path request under some conditions.
The thing I find weird is more the clients that are connecting, and then pumping out path
requests or announces at rates between 10 to 100 Hz, often sustained for 5-30 seconds.
Some of these clients connect for exactly 5 seconds (with a jitter of only ~5-15ms), and
start spamming as soon as they're connected, then disconnect (presumably to rotate IPs).
Most of that kind of "traffic" comes from TOR exit nodes.
Another class of them stay connected, and will wait for varying intervals until they
start dumping. They then dump their (primarily PR) spam for destinations that has never
existed on the network, keep it up for 10-30 seconds, go silent again and wait until the
next cycle.
Sometimes it coincides with several clients doing it at the same time, which can easily
compound into 10k+ path requests being received in around 30 seconds.
โ 3 โ 0 โค 1
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-15
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ KenAKAFrosty #15 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
โ Mark wrote:
โ Thanks for looking into it Ken, but it's not so much the announce waves happenning
โ as clients reconnect that I find weird. Those make pretty decent sense, and it's
โ usually just one or two announces per sub-interface. They do tend to come in a
โ wave, but usually only around a couple of hundred at a time, not really a problem.
โ
โ And active links that go down wouldn't cause new path requests when they drop,
โ not in RNS at least. It's only attempted links that were pending but never
โ established (and only at the initiator and penultimate hop that would occur, not in
โ any way over the entire path) that trigger a path request under some conditions.
โ
โ The thing I find weird is more the clients that are connecting, and then pumping
โ out path requests or announces at rates between 10 to 100 Hz, often sustained for
โ 5-30 seconds. Some of these clients connect for exactly 5 seconds (with a jitter of
โ only ~5-15ms), and start spamming as soon as they're connected, then disconnect
โ (presumably to rotate IPs). Most of that kind of "traffic" comes from TOR exit
โ nodes.
โ
โ Another class of them stay connected, and will wait for varying intervals until
โ they start dumping. They then dump their (primarily PR) spam for destinations that
โ has never existed on the network, keep it up for 10-30 seconds, go silent again and
โ wait until the next cycle.
โ
โ Sometimes it coincides with several clients doing it at the same time, which can
โ easily compound into 10k+ path requests being received in around 30 seconds.
Oh right sorry to clarify, I mean in LXMF, the rtt x 6 timeout. Assuming for these kinds
of links that's like 0.5-2s. With maybe a few seconds of delay from the polling timing.
But anyway, yeah what you're describing definitely sounds like it's a different breed.
Will keep an eye on that pattern going forward. For now at the very least it's one small
lil data point that I'm currently not seeing any sign of that traffic pattern from my
position.
Thanks!
โ 0 โ 0 โค 0
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-16
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ KenAKAFrosty #16 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Actually, might've spoken too soon; gonna dig in more on specifically looking for the
volume of destinations that i've not heard an announce for myself and work backward from
there. If anything fruitful comes of it I'll share with ya
โ 1 โ 0 โค 0
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-17
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ K8 #17 โ
โ 8e4525cda4482720 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
โ Mark wrote:
โ Thanks for the testing and feedback everyone!
โ
โ I just pushed another round of fixes and improvements to Aleph, latest on
โ `master` now. There was a few mistakes relating to PR processing, and a range of
โ other bugs fixed as well. I think some of the core pegging might have stemmed from
โ a stupid bug in the way pending PRs were processed, and that's fixed now.
โ
โ @K8 (and others), can you try the latest version (and also try dropping loglevel
โ below `LOG_PATHING`)? Under insane announce/PR loads, logging all of that is
โ going to have a significant performance hit, especially on IOPS constrained
โ servers. I recently refactored all the logging so `LOG_DEBUG` should now be
โ sufficient for mostly everything, unless you actually want to see all that
โ path/announce log spam.
Cool, I'm deployed up to the latest HEAD now, we'll see how it goes. I sent you a patch
to move logging into its own thread, that should help some with that issue (previously it
was also reopening and closing the logfile on every single log write, that certainly
wasn't helping anything either). I've thought about the logging perf before, but my
server has NVMe backed storage that I've tested to be plenty fast so I don't think that
is the issue I'm seeing. I've still reduced the log level for now to see if it helps. So
far it doesn't seem to have changed the CPU usage patterns at all.
More stats from the last version I deployed:
Totals : โ11.30 GB 1.51 Mbps
โ9.99 GB 1.01 Mbps
Qu. Pressure : 0.0% total, 0 pkts, 85120 dropped
0.0% data, 0 pkts
0.0% announce, 0 pkts, 40479 dropped
0.0% path request, 0 pkts, 290 dropped
0.0% ingress limiter, 0 pkts, 44351 dropped
Uptime is 20h, 49m and 56.32s, 573 entries in link table
โ 2 โ 0 โค 1
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-18
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ Mark #18 โ
โ 8dd57a7382268096 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Thanks for the patch @K8, got it fetched locally here. I'll look at that after 1.5.0 is
out, there's already plenty of new stuff for this one. To be honest, I'm a bit on the
fence about adding the buffering and and non-blocking log writes. Call it a work-related
injury, but I don't like log-files that lag ;) But let's see what we can do to improve it.
Yeah, if it's a physical server it shouldn't be much of a problem, but on some VPSes on
shared storage backend, all the individual, short and small file open/write/flush ops can
bog things down.
If you're on the latest source version, running "vanilla" with no local patches, it's
very odd that the sustained CPU spikes are still there. I'd really love to know what on
earth is causing that. Not seeing anything like it here on any of my nodes anymore. It is
the rnsd process itself doing it, right? Not some other RNS program?
If it's completely opaque, it might be worth adding a sprinkling of with
RNS.Profiler: contexts to various parts of Transport.py to capture a profiling trace
and guage where things are going wonky.
โ 1 โ 0 โค 0
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-19
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ K8 #19 โ
โ 8e4525cda4482720 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Fair enough about log buffering! I have it set up to notify the writer thread using more
or less the same kind of structure as the new inbound queues, and I have the file
buffering mode set to flush after every line, so in most cases it should be pretty
responsive. But I'm sure there's a middle ground solution to be found. At the very least
there are some more edge cases to handle like making sure that the log queue is fully
flushed before exiting.
Yup, just the rnsd process. Speaking of the Profiler, I just spent part
of the day reworking it so that it's possible to get live results through
rnstatus , which is way way more usable when trying to profile a big running node
like this where the problems don't usually start early on. It's still a bit shit and bogs
down the node when processing a ton of samples, so take these numbers (especially the
maximums) with a grain of salt:
โ 2 โ 0 โค 1
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
post-20
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ K8 #20 โ
โ 8e4525cda4482720 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โธ 27d ago
Profiling :
Transport._inbound
Samples : 1000000
Total : 696s 647ms 188.38ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 696.65ยตs | 381.61ยตs | 11.09ยตs | 940ms 672.49ยตs | 7ms 175.09ยตs )
1m : ( 716.2ยตs | 362.31ยตs | 14.6ยตs | 870ms 693.37ยตs | 8ms 878.46ยตs )
5m : ( 691.25ยตs | 373.71ยตs | 14.6ยตs | 884ms 220.88ยตs | 8ms 735.54ยตs )
30m : ( 718.95ยตs | 381.5ยตs | 11.09ยตs | 940ms 672.49ยตs | 8ms 418.49ยตs )
Transport._inbound.0qget
Samples : 1000000
Total : 2467s 848ms 6.58ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 2ms 467.85ยตs | 9.84ยตs | 1.31ยตs | 1s 66ms 644.86ยตs | 14ms 587.09ยตs )
1m : ( 1ms 710.88ยตs | 5.99ยตs | 1.42ยตs | 894ms 199.92ยตs | 15ms 219.51ยตs )
5m : ( 2ms 77.09ยตs | 7.15ยตs | 1.42ยตs | 894ms 199.92ยตs | 16ms 111.42ยตs )
30m : ( 2ms 547.86ยตs | 9.58ยตs | 1.31ยตs | 992ms 174.0ยตs | 16ms 830.31ยตs )
Transport._inbound.0qput
Samples : 1000064
Total : 18s 481ms 312.09ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 18.48ยตs | 13.08ยตs | 1.28ยตs | 80ms 85.21ยตs | 185.64ยตs )
1m : ( 33.14ยตs | 7.77ยตs | 1.36ยตs | 31ms 200.31ยตs | 542.1ยตs )
5m : ( 20.51ยตs | 9.6ยตs | 1.36ยตs | 31ms 200.31ยตs | 283.69ยตs )
30m : ( 18.87ยตs | 12.95ยตs | 1.28ยตs | 80ms 85.21ยตs | 199.28ยตs )
Transport._inbound.1preamble
Samples : 1000000
Total : 12s 72ms 308.7ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 12.07ยตs | 10.26ยตs | 2.23ยตs | 26ms 467.84ยตs | 35.06ยตs )
1m : ( 10.14ยตs | 7.94ยตs | 2.23ยตs | 5ms 276.61ยตs | 31.41ยตs )
5m : ( 10.82ยตs | 8.97ยตs | 2.23ยตs | 5ms 276.61ยตs | 18.34ยตs )
30m : ( 12.01ยตs | 10.24ยตs | 2.23ยตs | 9ms 16.73ยตs | 21.21ยตs )
Transport._inbound.2broadcast
Samples : 1000000
Total : 1s 769ms 213.77ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 1.77ยตs | 1.48ยตs | 0.51ยตs | 8ms 963.64ยตs | 14.58ยตs )
1m : ( 1.51ยตs | 1.25ยตs | 0.54ยตs | 111.46ยตs | 2.32ยตs )
5m : ( 1.62ยตs | 1.37ยตs | 0.54ยตs | 666.38ยตs | 3.59ยตs )
30m : ( 1.8ยตs | 1.49ยตs | 0.51ยตs | 8ms 963.64ยตs | 19.05ยตs )
Transport._inbound.3routing
Samples : 1000000
Total : 356s 111ms 111.75ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 356.11ยตs | 252.79ยตs | 0.84ยตs | 847ms 270.25ยตs | 3ms 31.4ยตs )
1m : ( 427.17ยตs | 246.73ยตs | 0.85ยตs | 763ms 679.62ยตs | 5ms 979.5ยตs )
5m : ( 379.65ยตs | 267.45ยตs | 0.85ยตs | 763ms 679.62ยตs | 4ms 596.7ยตs )
30m : ( 342.67ยตs | 221.09ยตs | 0.85ยตs | 847ms 270.25ยตs | 3ms 352.37ยตs )
Transport._inbound.4announce
Samples : 191543
Total : 159s 10ms 113.93ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 830.15ยตs | 370.23ยตs | 1.03ยตs | 1s 6ms 391.2ยตs | 10ms 522.28ยตs )
1m : ( 2ms 566.51ยตs | 349.36ยตs | 179.35ยตs | 1s 6ms 391.2ยตs | 28ms 595.39ยตs )
5m : ( 1ms 524.79ยตs | 358.61ยตs | 179.35ยตs | 1s 6ms 391.2ยตs | 20ms 627.92ยตs )
30m : ( 960.59ยตs | 376.37ยตs | 179.31ยตs | 1s 6ms 391.2ยตs | 13ms 753.1ยตs )
60m : ( 846.51ยตs | 369.09ยตs | 5.64ยตs | 1s 6ms 391.2ยตs | 10ms 910.17ยตs )
Transport._inbound.5linkreq
Samples : 139
Total : 656.32ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 4.72ยตs | 4.16ยตs | 2.26ยตs | 39.11ยตs | 3.61ยตs )
Transport._inbound.6localdata
Samples : 1000000
Total : 215s 707ms 890.47ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 215.71ยตs | 3.47ยตs | 0.8ยตs | 989ms 968.77ยตs | 5ms 657.0ยตs )
1m : ( 745.63ยตs | 2.83ยตs | 0.87ยตs | 989ms 968.77ยตs | 11ms 821.01ยตs )
5m : ( 379.16ยตs | 3.16ยตs | 0.87ยตs | 989ms 968.77ยตs | 8ms 654.97ยตs )
30m : ( 292.53ยตs | 3.52ยตs | 0.87ยตs | 989ms 968.77ยตs | 7ms 393.89ยตs )
60m : ( 220.61ยตs | 3.47ยตs | 0.87ยตs | 989ms 968.77ยตs | 5ms 766.42ยตs )
Transport._inbound.7proofs
Samples : 15031
Total : 8s 489ms 946.9ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 564.83ยตs | 534.0ยตs | 1.09ยตs | 402ms 697.03ยตs | 3ms 379.42ยตs )
1m : ( 483.94ยตs | 438.67ยตs | 1.44ยตs | 14ms 398.25ยตs | 808.8ยตs )
5m : ( 518.79ยตs | 505.57ยตs | 1.44ยตs | 14ms 398.25ยตs | 655.32ยตs )
30m : ( 564.51ยตs | 545.23ยตs | 1.3ยตs | 38ms 306.9ยตs | 780.41ยตs )
60m : ( 567.58ยตs | 535.1ยตs | 1.09ยตs | 402ms 697.03ยตs | 3ms 473.32ยตs )
Transport._outbound
Samples : 480707
Total : 378s 887ms 972.18ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 788.19ยตs | 198.96ยตs | 8.4ยตs | 3s 894ms 165.77ยตs | 12ms 22.41ยตs )
1m : ( 4ms 821.53ยตs | 208.63ยตs | 34.94ยตs | 2s 440ms 169.09ยตs | 57ms 900.53ยตs )
5m : ( 2ms 242.8ยตs | 207.01ยตs | 30.96ยตs | 3s 894ms 165.77ยตs | 22ms 632.99ยตs )
30m : ( 1ms 83.75ยตs | 205.03ยตs | 30.96ยตs | 3s 894ms 165.77ยตs | 14ms 789.18ยตs )
60m : ( 800.34ยตs | 199.09ยตs | 8.4ยตs | 3s 894ms 165.77ยตs | 12ms 149.74ยตs )
Transport.jobs
Samples : 14014
Total : 493s 468ms 695.57ยตs
Mean | Median | Min | Max | St. dev
Stats : (35ms 212.55ยตs | 44.41ยตs | 14.11ยตs | 1s 781ms 862.98ยตs | 148ms 344.6ยตs )
1m : (73ms 288.13ยตs | 43.7ยตs | 15.73ยตs | 1s 781ms 862.98ยตs | 257ms 256.36ยตs)
5m : ( 61ms 75.99ยตs | 41.55ยตs | 14.81ยตs | 1s 781ms 862.98ยตs | 226ms 485.52ยตs)
30m : (50ms 433.39ยตs | 43.63ยตs | 14.11ยตs | 1s 781ms 862.98ยตs | 193ms 717.94ยตs)
60m : (37ms 646.16ยตs | 43.82ยตs | 14.11ยตs | 1s 781ms 862.98ยตs | 154ms 789.88ยตs)
Transport.jobs.gc
Samples : 759
Total : 371s 241ms 487.19ยตs
Mean | Median | Min | Max | St. dev
Stats : (489ms 119.22ยตs | 576ms 382.83ยตs | 35ms 737.43ยตs | 1s 78ms 71.96ยตs | 240ms 796.72ยตs)
1m : (820ms 979.27ยตs | 807ms 622.03ยตs | 685ms 695.3ยตs | 1s 78ms 71.96ยตs | 93ms 50.71ยตs )
5m : (776ms 645.01ยตs | 755ms 954.83ยตs | 631ms 438.14ยตs | 1s 78ms 71.96ยตs | 77ms 317.52ยตs )
30m : (700ms 853.22ยตs | 695ms 284.06ยตs | 545ms 27.43ยตs | 1s 78ms 71.96ยตs | 76ms 137.25ยตs )
60m : (524ms 311.13ยตs | 605ms 417.47ยตs | 92ms 458.45ยตs | 1s 78ms 71.96ยตs | 216ms 533.47ยตs)
Transport.preprocess_inbound
Samples : 1000113
Total : 602s 916ms 166.08ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 602.85ยตs | 403.73ยตs | 0.79ยตs | 1s 105ms 270.83ยตs | 3ms 959.36ยตs )
1m : ( 1ms 207.85ยตs | 386.93ยตs | 0.82ยตs | 1s 105ms 270.83ยตs | 11ms 616.07ยตs )
5m : ( 837.96ยตs | 397.57ยตs | 0.81ยตs | 1s 105ms 270.83ยตs | 7ms 457.0ยตs )
30m : ( 642.77ยตs | 405.44ยตs | 0.79ยตs | 1s 105ms 270.83ยตs | 4ms 638.01ยตs )
Transport.transmit
Samples : 1952690
Total : 557s 122ms 313.25ยตs
Mean | Median | Min | Max | St. dev
Stats : ( 285.31ยตs | 206.69ยตs | 0.62ยตs | 851ms 483.43ยตs | 2ms 697.23ยตs )
1m : ( 244.3ยตs | 201.96ยตs | 0.62ยตs | 851ms 483.43ยตs | 2ms 168.61ยตs )
5m : ( 252.74ยตs | 202.99ยตs | 0.62ยตs | 851ms 483.43ยตs | 2ms 430.83ยตs )
30m : ( 273.97ยตs | 203.25ยตs | 0.62ยตs | 851ms 483.43ยตs | 2ms 652.99ยตs )
60m : ( 285.32ยตs | 206.7ยตs | 0.62ยตs | 851ms 483.43ยตs | 2ms 697.33ยตs )
โ 4 โ 0 โค 7
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Page 2 of 6
โ INFO โ Identify to this node to post. How?
bottom